A general approach for mutual information minimization and its application to blind source separation

In this paper, a non-parametric “gradient” of the mutual information is first introduced. It is used for showing that mutual information has no local minima. Using the introduced “gradient”, two general gradient based approaches for minimizing mutual information in a parametric model are then presented. These approaches are quite general, and principally they can be used in any mutual information minimization problem. In blind source separation, these approaches provide powerful tools for separating any complicated (yet separable) mixing model. In this paper, they are used to develop algorithms for separating four separable mixing models: linear instantaneous, linear convolutive, Post Non-Linear (PNL) and Convolutive Post Non-Linear (CPNL) mixtures. Index Terms Mutual Information, Information theoretic learning, Gradient of mutual information, Score Function Difference (SFD), Independent Component Analysis (ICA), Blind Source Separation (BSS), Convolutive mixtures, Post Non-Linear (PNL) mixtures, Convolutive Post Non-Linear (CPNL) mixtures,